The present invention relates to video compression technology and, more particularly, to scalable video coders.
Compressed video, which uses predictive coding algorithms and variable length coding, is sensitive to network impairments since these can cause error propagation. A single bit error or erasure can cause substantial degradation if no action is taken to stop or limit the extent of error propagation. Motion compensation allows the error to propagate both temporally and spatially. Because of this, there has been extensive effort in the video community to design new techniques that limit the extent of error propagation. However, almost all attempts to limit error propagation decrease the coding efficiency, some dramatically so. To ensure the best operation of the video coder in an error prone channel, the balance between resilience and efficiency must be managed carefully.
Scalable coding algorithms create a partitioning of the compressed bitstream into more and less important parts. This allows a natural combination with different mechanisms to prioritize network transport, for example, marking less important parts for early discard, applying unequal error protection, or facilitating rate matching between encoder and network. When used in conjunction with such techniques, scalable video can be very resilient to network introduced errors.
The propagation of enhancement-layer errors into the base-layer reconstruction is referred to herein as “drift” (this is in contrast to the more general term “error propagation” which is used herein to include the result of partial reception of the more important base-layer information). Early scalable video coders (like MPEG2 SNR scalability (SNRS)) allowed drift by using low priority enhancement-layer information to predict the high priority base-layer. However, in recent years, the development of scalable video encoders (like H.263 SNRS and spatial scalability (SS)) has focused on eliminating drift. In these algorithms, the base-layer is predicted only from the base-layer information. This strategy has been taken one step further in the development of MPEG4 Fine Granularity Scalability (FGS), in which the enhancement-layer information is also predicted only from base-layer information.
However, while recent scalable video coding algorithms are becoming more efficient at compressing the video, they lose compression efficiency because they ignore all enhancement-layer information when predicting the base-layer. In particular, recent experiments show that with MPEG2 SS, MPEG4 and H.263 scalability modes all suffer from 0.5-1.5 dB losses for every layer. FGS has particularly poor compression inefficiency because of its restricted prediction strategy.